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SECTION  I 
INTRODUCTION 

The  training  of  maintenance  skills  is  an  issue  of  major  concern 
to  the  military  due  to  increases  in  the  complexity  of  in-the-t'ield 
hardware  and  the  cost  of  training  maintenance  technicians.  Effective 
maintenance  training  programs  can  facilitate  full  utilization  of 
Hardware  technology  and  decrease  the  maintenance  costs  by  reducing  the 
number  of  technicians  required,  spare  parts  costs,  and  hardware  down 
time.  Traditionally,  hands-on  electronic  maintenance  training  has 
utilized  actual  equipment  trainers  (AETs)  to  provide  troubleshoot  ing 
training  to  the  component  level.  The  primary  objective  of  this  train¬ 
ing  has  been  to  develop  and  sustain  applied  skills  and  knowledge  in 
basic  electricity  and  electronics.  The  Basic  Electricity  and 
Electronics  (BE&E)  School,  located  at  the  Naval  Training  Center  in 
Orlando,  Florida,  utilizes  a  computer  managed  course  of  instruction 
that  empLoys  hands-on  maintenance  training  in  conjunction  with  self- 
paced  instructional  texts  to  train  basic  troubleshooting  concepts  and 
skills. 

A  recent  study  conducted  by  McDonald  &  Associates,  Inc. 

(McDonald,  Waldrop,  &  White,  1982)  at  the  BE&E  School  revealed  that 
students  did  not  always  utilize  optimum  troubleshooting  procedures. 
Efficient  electronic  troubleshooting  requires  isolation  of  the  faulty 
component  by  taking  readings  at  logical  test  points  and  using  the 
information  from  those  readings  to  determine  the  next  logical  test 
point.  A  review  of  the  School's  curriculum  indicated  that  logical 
troubleshooting  behavior  was  addressed,  but  not  stressed,  and  as  a 
result,  students  adopted  many  different  combinations  of  troubleshoot¬ 
ing  strategy  in  actual  hands-on  performance  tests.  The  combinations 
adopted  did  not  always  lead  to  optimum  troubleshooting  performance  and 
many  times  resulted  in  inefficient  test  point  probes. 

The  use  of  computer  assisted  instruction  (CAI)  that  interfaces 
simultaneously  with  an  interactive  video  can  optimize  the  costs  and 
time  associated  with  maintenance  skills  training.  Many  of  the  ben¬ 

efits  derived  from  CAI  are  directly  applicable  to  a  maintenance  train¬ 
ing  program  like  the  one  currently  used  at  BE&E.  CAI  can  provide 
practices,  reviews,  and  performance  tests  directly  to  the  student, 
freeing  the  Learning  Supervisor  (LS)  to  perform  other  instructional 
functions.  Student  performance  can  also  be  timed,  evaluated,  and 
remediated  by  the  computer.  Computer  models,  simulating  actual  equip¬ 
ment,  reduce  the  number  of  AETs  and  circuit  boards  needed,  thus 
reducing  equipment  costs.  Test  point  probing  and  assignment  of  faulty 
circuits  are  done  by  computer,  eliminating  deadtime  due  to  the  servic¬ 
ing  of  damaged  actual  equipment  and  circuit  boards.  In  addition,  CAI 
allows  the  constant  up-date  of  instructional  information  without 

costly  text  rewrites.  CAI  has  the  potential  to  provide  optimum 
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individualized  maintenance  training,  as  well  as  to  cut  costs  by  manag¬ 
ing  the  course  curriculum. 

To  test  the  applicability  of  CAI  to  the  BE&E  School,  a  pilot 
study  was  conducted  using  an  off-the-shelf  computer/video  assisted 
instructional  program  on  strategic  troubleshooting.  The  course 
emphasized  taking  a  logical  sequence  of  tests,  based  on  both  good  and 
bad  inputs  and  outputs,  to  localize  the  faulty  component  with  the 
least  number  of  testprobes.  Twelve  students  participated  in  the  study 
(four  experimental  and  eight  controls).  Each  experimental  subject  was 
matched  with  a  male  and  female  control  subject.  Results  from  the 
study  indicated  that  the  experimental  subjects  spent  less  time  trouble¬ 
shooting  a  faulty  board,  took  fewer  probes,  and  spent  less  time  com¬ 
pleting  the  BE&E  Electronic  Technician  (ET )  curriculum.  Results  from 
the  pilot  study  were  encouraging  but  difficult  to  generalize  to  the 
overall  population,  due  to  the  small  sample  size.  In  order  to  deter¬ 
mine  whether  or  not  the  course  was  effective,  further  research  was 
required  utilizing  a  larger  sample  size,  with  appropriate  control 
conditions. 

Methods  similar  to  those  in  the  pilot  study  were  used  in  the 
current  research  to  implement  the  course  just  prior  to  students  enter¬ 
ing  the  ET  Splice  modules  of  BE&E  School.  The  purpose  of  the  study 
was  to  examine  the  effectiveness  of  the  troubleshooting  CAI  course  on 
troubleshooting  behavior  during  performance  tests  in  the  ET  Splice 
phase  of  instruction.  The  hypotheses  to  be  tested  were: 

a.  Students  participating  in  the  troubleshooting  CAI  will  trou¬ 
bleshoot  morn  efficiently  than  control  students. 

b.  High  proficiency  students  will  troubleshoot  more  efficiently 
than  medium  and  l^ow  proficiency  students,  and  medium  profi¬ 
ciency  students  will  perform  more  efficiently  than  low  profi¬ 
ciency  students. 

c.  High  proficiency  students  will  complete  the  ET  Splice  curric¬ 
ulum  in  fewer*  hours  than  medium  and  low  proficiency  students, 
and  medium  proficiency  students  will  complete  the  ET  Splice 
curriculum  in  fewer  hours  than  low  proficiency  students. 

d.  Students  participating  in  the  troubleshooting  CAI  will  com¬ 
plete  the  ET  Spiice  curriculum  in  fewer  hours  than  control 
students  . 
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SECTION  II 


MtiThOD 

The  primary  objective  of  the  research  program  was  to  determine 
tiie  effects  of  the  strategic  troubleshoot  ing  course  on  student  troubK - 
shoot  ing  efficiency  on  a  typical  trouDleshoot  ing  task  of  the  ET  Splice 
course.  Since  the  purpose  of  the  study  was  to  determine  the  trains  tor 
ot  training  of  the  troubleshooting  CAI  to  actual  hand c -on  perfor¬ 
mance  ,  the  research  was  designed  to  be  integrated  into  the  testing 
procedures  used  at  t!'e  BENE  School.  The  CAI  workbook  and  program  were 
ex  tens i ve  ly  modified  to  present  the  student  with  curriculum  similar  to 
that  used  at  the  BENE  School  and  to  fulfill  r.  search  data  require¬ 
ments.  The  BENE  School  was  used  as  a  model  d inc e  the  results  of  this 
study  were  intended  for  use  by  curriculum  designers  of  military  elec¬ 
tronics  training  with  additional  application  to  industrial  training 

poss i ble . 

APPARATUS 

STRATEGIC  TROUBLESHOOTING  COURSEWARE.  The  experimental  treatment 
consisted  of  an  off-the-shelf  strategic  troubleshooting  course.  This 
course  combined  videotape  presentations,  workbook  exercises,  and 
computer-assisted  instruction  (CAI)  materials.  The  computer  graphical¬ 
ly  presented  hypothetical  circuits  with  bad  outputs  and  allowed  the 
student  to  select  test  points  and  see  the  results  of  the  tests.  Tne 
computer  provided  feedback  on  whether  or  not  a  proper  troubleshooting 
strategy  was  being  used.  The  principal  troubleshooting  strategy 
taught  by  this  course  is  the  half-split  technique,  which  involves 
successive  testing  ot  the  midpoint  between  known  good  and  bad  signals 
until  the  fault  is  isolated.  The  program  presentation  time  was  a 
minimum  of  9  hours,  and  additional  time  was  required  when  students 
repeated  units,  reviewed  practice  problems,  or  required  additional 

e lar i f  icat  ion . 

BASIC  COURSEWARE.  i5  CAI, /video  program  which  taught  the  BASIC  computer 
language  was  developed  for  use  with  the  control  subjects.  The  course 
was  designed  to  be  similar  in  length  and  instructional  characteristics 
to  the  troubleshooting  CAI,  in  order  to  make  the  course  appear  rel¬ 
evant  to  the  control  subjects,  while  avoiding  any  material  which  might 
be  directly  applicable  to  troubleshooting.  The  video  portion  entitled 
"Computer  Programming:  BASIC  for  Microcomputers"  was  made  available 

from  Educational  Activities,  Inc.  and  was  integrated  with  a  TRS-60 
Model  III  BASIC  computer  interactive  course. 

CAI  EQUIPMENT.  Two  TRS-80  Model  III  computers  and  2  Betamnx  video 

playback  units  with  video  monitors  were  used  to  present  the  trouble¬ 

shooting  CAI.  Headphones  were  utilized  to  prevent  inter ference  during 
simultaneous  operation  of  the  two  stations.  The  treatment  control  CAI 
(BASIC  course)  utilized  the  same  equipment  ns  the  t roub les hoot  ing  CAI. 
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j  :<Oi‘  l'l  N(J  !•'()  IJ  l  I'MEN  T .  Since  complexity  tin-  L  roub  1  csimot  i  :ig 

t.i  .  k  wa certain  to  at  feet  student  behavior,  3  .ii  f  ferent  print,  d 
circuit  boards  were  utilized  tor  the  collection  of  actual  troubleshoot  - 
in  per  tor mu nee  data:  a  simple  205-5  Second  IF  Ampl  if  ier  (Second  IF), 
a  medium  complexity  205-4  First  IF  Amplifier  (First  IF)  and  a  highly 
complex  Power  Supply  (Power  Supply)  board  with  feedback  loops  These 
boards  were  contained  in  a  NIOA  Model  205  Transceiver  Trainer  and  a 
N  I  DA  Model  201  Power  Supply  Trainer  utilized  as  a  normal  part  of  the 
curriculum  in  the  ET  Splice  course. 

The  study  utilized  9  prefaulted  boards  for  each  of  the  3  printed 
circuit  boar i  types,  providing  27  prefaulted  boards.  Boards  were 
prefaulted  by  the  manufacturer .  The  9  faulted  boards  for  each  board 
type  were  divided  into  3  fault  groups,  based  on  fault  difficulty. 
This  .  1  1  owed  random  assignment  of  faults  to  each  student,  to  prevent 
the  possil  lity  of  prior  student  knowledge  of  fault  location  and  to 
reduce  performance  variance  due  to  fault  difficulty  differences. 

A  total  of  4  trainers  (2  201  Power  Supply  Trainers  and  2  205 

Transceivers)  were  available,  thus  allowing  any  combination  of  2  sep¬ 
arate  performance  tests  to  be  observed  at  one  time.  Additional  trou¬ 
bleshooting  equipment  included:  2  sweep  generators,  2  oscilloscopes, 

2  Simpson  Multimeters,  and  various  probes.  Any  additional  equipment 
required  was  supplied  by  the  School,  and  equipment  and  circuit  boards 
were  maintained  by  the  School. 

SUBJECTS 

Subjects  were  selected  from  students  enrolled  in  the  ET  Splice 

program  Modules  30  to  j4  at  the  BE&E  School.  These  modules  are  a 
preparatory  -ourse  for  an  Electronic  Technician  rating.  Students  were 
male,  E3  Seamen  ranging  in  age  from  17  to  35  with  the  average  ape 
being  19.  The  education  level  ranged  from  completion  of  high  scho  ’ 
to  1  year  of  college. 

All  students  were  tracked  prior  to  entering  ET  Splice  using  the 
School's  computer  managed  instruction  (CMI )  printouts.  This  allowed 
the  researcher  to  predict  when  students  would  be  entering  ET  Splice 

and  readv  to  be  assigned  to  1  of  the  3  treatment  condition^. 

Each  student  was  assigned  roficiency  level  of  high,  medium  or 
low  based  on  their  actual  elapsed  time  listed  on  the  daily  CMI  print¬ 
outs.  This  time  represents  a  student's  total  contact  time,  accumulat¬ 
ed  from  the  time  the  student  entered  the  BE&E  curriculum  to  just  prior 
to  entering  ET  Splice. 
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Proficiency  levels  were  hypothesized  to  significantly  influence 
troubleshooting  behavior.  Student  proficiency  categories  had  been 
determined  during  previous  research  (McDonald,  et.al.,  1982)  by  look¬ 
ing  at  a  random  sample  of  225  student  BE&E  School  completion  times. 
Proficiency  categories  were  determined  by  monitoring  completion  times 
on  BE&E  CMI  printouts  and  dividing  the  range  of  times  into  3  equal 
groups  of  75  each.  This  resulted  in  the  following  proficiency  levels: 
high  proficiency  0-224.99  hours,  medium  225-289.99  hours,  and  low 
290-305.99  hours.  However,  between  the  previous  research  and  the 
current  research,  changes  in  the  School's  curriculum,  physical  Lay¬ 
out  ,  and  student  population  resulted  in  a  change  in  the  tine  distribu¬ 
tion  of  student  hours.  Using  the  same  process  and  the  existing 
categories  as  a  base,  new  3E&E  completion  dimes  were  monitored,  and 
the  resulting  categories  used  to  reclassify  student  proficiency  were: 
high  Less  than  212  hours,  medium  21 2—31 i . 99  hours,  and  low  312-411.99 
hours  . 

ET  Splice  students  were  randomly  selected  after  being  tracked 
through  the  CMI  data  and  classified  ir.  1  of  the  3  proficiency  levels. 
Six  students  from  each  of  the  3  proficiency  levels  were  assigned  to  1 
of  the  3  treatment  conditions.  A  total  of  54  subjects  were  used  in 
repeated  measures  across  all  3  circuit  board  types  (18  troubleshooting 
CAI,  18  control  CAI,  and  18  no-treatment  controls). 

Fifty-four  students  took  a  total  of  162  performance  tests  across 
all  3  boards.  A  total  of  54  performance  tests  (6  at  each  of  the  3 
proficiency  leveLs  on  each  of  the  3  types  of  boards)  were  observed  for 
each  of  the  treatment  conditions.  Experimental  matrices  were  used  to 
assure  chat  all  treatment  conditions  were  balanced  and  completely 
randomized.  Student  attrition  occurred  periodically  due  to  equipment 
malfunction,  student  double-shifting  because  of  transfer  orders,  or 
reclassification  out  of  the  ET  Splice  curriculum  into  an  Electronics 
Warfare  program.  This  had  no  effect  on  finishing  the  complete  repeat¬ 
ed  measures  since  these  students  were  replaced  by  others  during  the 
time  set  aside  for  data  collection. 

PROCEDURE 

The  equipment  for  the  presentation  of  the  troubleshooting  CAI  and 
control  CAI  conditions  was  set  up  at  the  Orlando  Naval  Training 
Equipment  Center's  Human  Factors  Laboratory.  Two  stations  were  avail¬ 
able  with  only  l  type  of  treatment  condition  run  at  one  time,  i.e.,  2 
treatment  control  students  or  2  experimental  treatment  students. 
Headphones  were  used  to  prevent  the  2  stations  from  interfering  with 
fine  another.  A  researcher  was  provided  by  the  Human  Factors  Laboratory 
to  monitor  the  2  CAI  training  stations  and  to  administer  tests.  Two 
stations  were  also  set  up  at  the  BE&E  School  for  collection  of  criteri¬ 
on  performance  data. 
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A  pilot  subject  was  sent  to  the  Human  Factors  Laboratory  to 
participate  in  the  troubleshooting  CAI.  After  completing  the  course, 
the  student  returned  to  the  BE&E  School  and  was  observed  on  1  Power 
Supply  and  2  IF  Amplifier  performance  tests.  This  preliminary  data 
provided  the  on-site  researcher  with  information  on  the  CAI  program 
timing,  student  acceptance  of  the  CAI,  and  any  possible  BE&E  curric¬ 
ulum  interference.  In  addition,  it  allowed  the  researcher  to  begin 
tracking  students  at  BE&E  School,  to  assign  proficiency  levels  and 
treatment  conditions. 

Students  assigned  to  the  troubleshooting  CAI  or  control  CAI  were 
sent  to  the  Human  Factors  Laboratory  and  fold  to  report  there  for  the 
next  few  days  instead  of  reporting  to  BE&E  School.  These  students 
were  put  on  temporary  hold  on  the  School's  CMI  system  so  the  2  to  3 
class  days  spent  participating  in  the  CAI  treatment  condition  would 
not  affect  their  class  standing.  After  students  completed  their 
assigned  CAI  condition,  they  returned  to  BE&E  and  proceeded  with  their 
normal  ET  Splice  curriculum.  The  LS  sent  all  BE&E  students  to  the 
research  station  when  they  were  ready  for  performance  tests  on  Module 
30-2  (Power  Supply)  or  Module  31-3  (Transceiver).  This  allowed  the 
researcher  to  observe  performance  tests  from  the  students  who  partici¬ 
pated  in  the  CAI  courses,  as  well  as  to  randomly  select  no-treatment 
control  students.  Eighteen  troubleshooting  CAI  students,  18  control 
CAI,  and  18  no-treatment  control  students  were  observed  at  the 
researcher's  station.  Three  performance  tests  were  observed  for  every 
student,  1  on  the  Power  Supply,  1  on  the  First  IF  Amplifier  and  1  on 
the  Second  IF  Amplifier,  totalling  162  performance  tests.  A  pre¬ 
faulted  circuit  board  was  randomly  selected  from  the  appropriate  fault 
difficulty  group  for  each  student  by  the  researcher  before  each  perfor¬ 
mance  tes  t . 

The  ET  Splice  curriculum  is  a  self-paced  program.  Students 
participating  in  the  research  program  took  their  performance  tests  in 
normal  sequence,  without  affecting  their  normal  course  workload  or 
hours.  The  only  modification  was  that  3  of  their  performance  tests 
were  taken  at  the  research  station  using  a  circuit  board  which  was 
assigned  by  the  researcher,  rather  than  by  an  LS .  The  ET  Splice 
curriculum  utilizes  3  different  trainers,  administering  7  practice 
exercises  and  7  performance  tests,  on  7  different  printed  circuit 
boards.  The  research  data  collected  represents  3  performance  tests  on 
2  of  che  3  trainers.  The  average  ET  Splice  completion  time  is  60 
classroom  hours,  and  the  typical  class  day  runs  6  hours. 

Before  taking  performance  tests,  students  were  briefed  and 
assured  that  data  collected  would  not  affect  their  class  standing. 
Students  used  the  School's  Troubleshooting  Performance  Response  Sheets 
when  taking  the  performance  tests.  The  student  informed  the  research¬ 
er  when  the  fault  was  diagnosed.  The  student  then  took  the  response 
sheet  to  the  LS  for  feedback  on  whether  or  not  the  diagnosis  was 
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correct.  If  incorrect,  the  student  returned  to  the  research  station 
to  i.'imtinue  troubleshooting  the  same  faulty  circuit  until  the  cotrect 
fault  was  diagnosed.  After  the  correct  fault  was  diagnosed,  tin- 
student  returned  the  completed  response  sheet  to  the  researcher.  If 
the  performance  test  was  on  the  Power  Supply,  the  students  returned  to 
the  regular  BE&E  curriculum  until  Module  31-3  when  they  were  again 
referred  by  the  LS  to  the  research  station.  Prefaulted  circuit  boards 
on  the  NIDA  205  trainer  were  issued  in  random  sequence;  thus,  students 
could  receive  either  a  faulty  First  IF  Amplifier  or  a  faulty  Second  IF 
Amplifier  as  their  first  205  trainer  fault  card  and  could  receive  the 
remaining  one  as  their  second  performance  test  measure.  Students 
tilled  out  response  sheets  for  every  performance  test  taken  and  again 
went  to  the  LS  for  feedback  on  their  fault  diagnosis.  All  response 
sheets  were  returned  to  the  researcher  after  the  correct  fault  was 
d i agnosed . 

After  students  completed  all  3  performance  tests  at  the  research¬ 
er's  testing  area,  their  daily  progress  a‘t  the  School  was  monitored  on 
the  CMI  to  obtain  student  response  histories  after  they  completed  the 
final  BE&E  School  test.  Student  response  histories  provided  the 
researcher  with  each  student's  total  3E&E  School  completion  time. 

During  the  3  performance  tests  taken  at  the  research  station  by 
each  of  the  54  students,  the  researcher  recorded  the  dependent  perfor¬ 
mance  measures  of:  specific  test  points  taken,  total  number  of 
probes,  total  probe  time,  fault  diagnosed,  student  comments,  number  of 
trips  to  the  LS  and  any  additional  relevant  data.  The  162  performance 
tests  represent  the  criterion  measure  to  determine  any  transfer  of 
training  effects  from  the  experimental  treatment  to  actual  hands-on 
performance  tests  and  any  effect  on  troubleshooting  behavior  due  to 
proficiency  level.  In  addition,  each  student's  overall  BE&E  course 
completion  time  was  analyzed  to  determine  any  effect  due  to  treatment 
or  proficiency  level.  Data  from  these  analyses  are  discussed  in 
detail  in  the  Results  section. 

EXPERIMENTAL  DESIGN 

The  two-dimensional  design  matrix  was  a  3  (Treatment  Condition) 
by  3  (Proficiency  Level)  design  with  3  replications  across  circuit 
boards.  The  main  independent  variables  were  the  treatment  conditions 
(experimental  treatment,  treatment  control  and  baseline  control). 
Proficiency  level  was  used  as  a  blocking  variable,  assigning  subjects 
to  1  of  3  predetermined  categories  (blocks)  according  to  each  individ¬ 
ual's  total  number  of  hours  in  the  BE&E  curriculum  prior  to  ET  Splice. 
Assigning  a  specific  number  of  subjects  from  each  proficiency  level  to 
each  treatment  removes  a  large  source  of  potential  variation  that 
might  have  occurred  had  proficiency  levels  been  assigned  randomly  to 
treatment  conditions.  Fault  groups  and  proficiency  levels  were 
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matched  across  all  3  treatment  conditions  to  diminish  the  effects  of 
extraneous  variance  and  control  for  individual  differences. 

The  experimental  design  studied  the  effects  of  the  troubleshoot¬ 
ing  CAI  course  versus  two  types  of  control  groups.  The  treatment 
concrol  group  was  employed  to  account  for  any  Hawthorne  effect  on 
student  performance  which  might  result  from  changes  in  the  student's 
normal  instructional  environment.  The  treatment  control  manipulated 
the  student's  environment  and  the  sequence  of  learning  so  that  stim¬ 
ulus  conditions  closely  corresponded  to  those  of  the  experimental 
treatment  group  (troubleshooting  CAI).  The  baseline  (no- treatment ) 
control  group  was  used  to  compare  the  effects  of  the  troubleshooting 
CAI  against  the  regular  class  curriculum,  as  well  as  to  examine  the 
possible  Hawthorne  effects  of  removing  the  student  from  the  BE&E 
environment.  The  matrix  represents  an  independent  design  in  that  a 
group  of  students  experience  only  one  treatment  condition  across  all  3 
boards.  The  experimental  design  is  represented  in  Figure  1. 

The  dependent  variables  under  study  were  probe  time,  number  of 
test  points  probed,  correctness  on  first  fault  diagnosis,  and  total 
number  of  hours  to  finish  BE&E  School.  In  addition  to  analysis  on  the 
main  dependent  variables,  the  troubleshooting  logic  used  by  students 
was  examined.  All  data  were  collected  on  3  performance  tests  taken  by 
each  student  on  the  3  circuit  boards.  Analysis  of  these  data  indicat¬ 
ed  the  degree  of  transfer  of  training  from  the  treatment  conditions. 

A  two-way  Analysis  of  Variance  (ANOVA)  procedure  was  used  to 
analyze  the  main  effects  of  the  design  matrix  (Ferguson,  1976),  repli¬ 
cated  across  the  3  circuit  board  types.  This  analysis  allowed  the 
simultaneous  examination  of  both  the  independent  and  combined  effects 
of  treatment  conditions  $nd  proficiency  levels.  Each  student  was 
classified  in  1  proficiency  level  and  exposed  to  1  set  of  experimental 
conditions  across  all  3  boards.  However,  since  the  effect  of  differ¬ 
ences  between  the  3  board  types  was  not  a  primary  research  question, 
the  boards  were  analyzed  as  independent  designs.  The  2  CAI  treatment 
conditions  (experimental  and  control)  were  examined  for  any  variabil¬ 
ity  in  troubleshooting  performance  when  compared  to  the  no-treatraent 
control  group. 

Dichotomous  data  were  analyzed  using  a  Chi-Square  test  (Siegel, 
1956).  These  data  included  whether  or  not  the  student  was  correct  on 
the  first  fault  diagnosis  attempt  and  the  number  of  troubleshooting 
strategies  used  pr.ior  to  the  first  fault  diagnosis.  This  analysis 
allowed  examination  of  the  frequency  of  correctness  on  first  fault 
diagnosis  and  number  of  strategies  used,  between  the  treatment  condi¬ 
tions.  In  addition,  Chi-Square  procedures  were  used  to  compare  suc¬ 
cess  of  Half-Split  troubleshooting  procedure  versus  all  other 
strategies  combined  on  the  first  fault  diagnosis. 
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PROFICIENCY  LEVEL 


TREATMENT  CONDITIONS 

HIGH 

MEDIUM 

LOW 

Troubleshooting  CAI 

1 

Control  (BASIC)  CAI 

No-Treatment  Control 

Figure  1.  Experimental  design  matrix. 
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Analysis  of  Variance  procedures  were  used  Co  look  at  differences 
between  treatment  conditions.  The  level  for  significant  difference 
was  .05,  i.e.,  there  must  be  a  95%  probability  that  the  difference  is 

no t  due  to  chance.  Analysis  of  Variance  procedures  only  indicated 
that  there  were  significant  differences  between  independent  variables. 
In  order  to  ascertain  where  the  significant  differences  were  occur¬ 
ring,  a  Fisher  Least  Significant  Difference  (LSD)  post  hoc  procedure 
(Wilkowitz,  ec  al.,  1976)  was  performed  on  all  significant  ANOVA  F 
tests.  This  compared  all  possible  paired  means  using  the  Mean  Square 
Within  as  the  population  variance  estimate.  Again,  the  probability 
level  for  significance  was  .05. 
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SECTION  111 


RESULTS 


ANALYSIS  OF  VARIANCE  -  MAIN  EFFECTS 

The  results  discussed  in  this  section  examine  criterion  perfor¬ 
mance  differences  between  students  in  the  experimental  treatment,  the 
control  treatment  and  the  no-treatment  (baseline)  control  conditions. 
The  primary  measures  of  effectiveness  were  the  number  of  probes  and 
the  time  taken  to  locate  the  fault  during  fault  isolation  on  the 
criterion  boards.  The  experimental  and  control  conditions  were  exam¬ 
ined  within  the  3  board  types  and  analyzed  as  Separate  ANOVA  designs. 

POWER  SUPPLY  BOARD.  The  ANOVA  totals  and  summary  data  lot  number  of 

points  probed  are  shown  in  Tables  1  and  2.  These  data  indicate  a 

significant  (p<.05)  performance  difference  between  treatment  condi¬ 
tions.  Scudent  proficiency  level,  however,  did  not  s igni f ican c  iy 
affect  the  number  of  points  probed. 

The  least  significant  difference  (LSD)  post  hoc  technique  was 
applied  to  determine  exactly  which  variable  differences  were  signif¬ 
icant.  The  LSD  uses  the  smallest  value  which  can  be  considered  signif¬ 
icant.  This  technique  examines  all  the  pair-wise  mean  differences 
within  a  variable  (e.g.,  treatment  conditions)  to  determine  which 
difference  is  the  source  of  significance.  Table  3  contains  the  mean 
data  for  the  treatment  conditions.  The  LSD  indicated  a  significant 
( p< . 05 )  difference  in  the  number  of  points  probed  between  the  baseline 

control  group  and  the  troubleshooting  CAI  group  (experimental  treat¬ 

ment),  with  the  control  group  probing  fewer  points. 

The  .ANOVA  totals  and  summary  data  contained  in  Tables  4  and  5 
indicated  a  significant  (p<.05)  effect  due  to  treatment  condition  on 
the  time  taken  to  isolate  the  fault  (probe  time  in  minutes).  Table  6 
contains  the  mean  probe  time  for  the  Power  Supply  board  by  treatment 
condition.  The  LSD  post  hoc  analysis  indicated  that  the  baseline 
control  group  took  significantly  (p<.05)  less  time  to  locate  the  fault 
than  the  troubleshooting  CAI  group..  Student  proficiency  level  did  not 
s ign i l icant iy  affect  the  amount  of  time  required  to  isolate  the  fault. 

FIRST  IF  BOARD.  The  ANOVA  totals  and  summary  data  for  points  probed 
on  the  First  IF  boat'd  are  contained  in  Tables  7  and  8.  The  ANOVA 
results  indicated  a  significant  (p<.05)  effect  due  to  student  profi¬ 
ciency  level.  Table  9  contains  the  mean  number  of  points  probed  by 
proficiency  level.  The  LSD  post  hoc  analysis  indicated  a  significant 
( p< .05)  difference  between  high  and  medium  proficiency  students  and 
between  high  and  low  proficiency  students  with  the  higher  proficiency 
subjects  probing  fewer  points.  The  treatment  conditions  did  not  have 
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TABLE  1.  POWER  SUPPLY  BOARD  -  ANOVA  TOTALS  - 
NUMBER  OF  POINTS  PROBED 


TREATMENT  CONDITION 

PROFICIENCY  LEVEL 

HIGH  MEDIUM  LOW 

Troubleshooting  CAI 

183 

502 

349 

Control  CAI 

303 

173 

389 

Control 

161 

146 

144 

TABLE  2.  POWER 

SUPPLY  BOARD  - 

ANOVA  SUMMARY 

-  NUMBER  OF  POINTS 

PROBED 

VARIATION 

SUM  OF 

DEGREES 

VARIANCE 

SOURCE 

SQUARES 

FREEDOM 

ESTIMATE 

F 

Prof  ic iency 

1652.18 

2 

826.09 

.54 

Treatment 

9997.06 

2 

4998.53 

3.29* 

Interac  t ion 

10803.16 

^4 

2700.79 

1.78 

Error 

68310.90 

45 

1518.02 

Total 

90763.56 

53 

NOTE:  *  p< . 05 

» 

t 

• 

TABLE  3.  POWER  SUPPLY  BOARD  -  TREATMENT  CONDITION  - 
MEAN  POINTS  PROBED 


TREATMENT  CONDITION 

MEAN  POINTS  PROBED 

! 

Troubleshooting  CAI 

57.44 

Control  CAI 

48.06 

Control 

25.06 
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TABLE  4.  POWER  SUPPLY  BOARD  -  ANOVA  TOTALS  - 
PROBE  TIME  (MINUTES) 

PROFICIENCY  LEVEL 


TREATMENT  CONDITION 

HIGH 

MEDIUM 

LOW 

T r ou b  1  es  hoo t  i ng  CA I. 

360 

374 

40  l 

Control  CAI 

724 

265 

40  1 

Control 

L  99 

133 

234 

TABLE  5.  POWER  SUPPLY  BOARD  -  ANOVA  SUMMARY  -  PROBE  TIME  (MINUTES) 


VARIATION 

SOURCE 

SUM  OF 
SQUARES 

DEGREES 

FREEDOM  *- 

VARIANCE 

ESTIMATE 

F 

P ro r ic  ienc y 

1350.52 

2 

675.26 

.  64 

Front meat 

7967.82 

2 

3983.91 

3.78* 

Interaction 

571.36 

4 

142.84 

.14 

Error 

47432.25 

45 

1054.05 

Tot  iL 

57322.15 

53 

NOTE:  *p< .05 

TABLE  6. 

POWER  SUPPLY 

BOARD  -  TREATMENT 

CONDITION  - 

MEAN  PROBE  TIME  (MINUTES) 


TREATMENT  CONDITION 

MEAN  TIME 

Troubleshooting  CAI 

63.06 

Control  CAI 

55.00 

Control 

34.22 
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TABLE  7.  FIRST  IF  BOARD  -  ANOVA  TOTALS  - 
NUMBER  OF  POINTS  PROBED 

PROFICIENCY  LEVEL 


TREATMENT  CONDITION 

HIGH 

MEDIUM 

LOW 

Troubleshooting  CAI 

142 

289 

222 

Control  CAI 

143 

291 

285 

Control 

88 

156 

195 

TABLE  8.  FIRST  IF  BOARD  -  ANOVA  SUMMARY  -  NUMBER  OF  POINTS  PROBED 


VARIATION 

SOURCE 

SUM  OF 
SQUARES 

DEGREES 

FREEDOM 

VARIANCE 

ESTIMATE 

F 

Profit iency 

4466.00 

2 

2233.00 

3.72* 

Treatment 

2380.58 

2 

1190.29 

1.98 

Interact  ion 

656.00 

4 

164.00 

.27 

Error 

26984.70 

45 

599.66 

Total 

34487.63 

53 

NOTE:  *p< .05 

TABLE  9.  FIRST  IF  BOARD  -  PROFICIENCY- 
MEAN  POINTS  PROBED 


PROFICIENCY 

MEAN  POINTS  PROBED 

High 

20.72 

Medium 

40.89 

Low 

39.00 

18 


NAVTRAEQUIPCEN  82-C-01 1 9-1 


a  significant  effect:  on  the  number  of  points  probed  on  the  First  IF 
board . 

Tables  10  and  11  contain  the  ANOVA  totals  and  summary  data  for 
the  probing  time  on  the  First  IF  board.  The  ANOVA  results  indicated 
no  significant  (pT.05)  performance  effects  due  to  treatment  conditions 
or  proficiency  levels  on  probe  time. 

SECOND  IF  BOARD.  Tables  12  and  13  contain  the  ANOVA  data  for  number 
of  points  probed  on  the  Second  IF  board.  These  data  indicate  no 
significant  effects  due  to  treatment  condition  or  proficiency  level. 

The  ANOVA  results  on  probe  time  on  the  Second  IF  board  indicated 
a  significant  (pT.05)  effect  due  to  treatment  condition  (Tables  14  and 
15).  The  LSD  post  hoc  analysis  indicated  that  the  troubleshooting  CAI 
group  and  the  baseline  control  'group  required  significantly  ( p<  .05 ) 
less  time  to  isolate  the  fault  than  the  control  (BASIC)  CAI  group 
(Table  16)  . 

CHI-SQUARE  ANALYSIS  -  TROUBLESHOOTING  SUCCESS 

One  measure  of  the  effectiveness  of  a  student’s  troubleshooting 
performance  is  correct  fault  isolation  on  the  first  fault  diagnosis 
attempt.  If  the  training  effects  of  the  experimental  treatment  and 
control  conditions  were  equal,  then  we  would  expect  the  number  of 
students  who  had  correctly  diagnosed  the  fault  on  the  first  diagnosis 
attempt  to  be  equal  across  conditions.  Whether  or  not  a  student  was 
correct  on  the  first  fault  diagnosis  is  a  dichotomous  variable  which 
can  be  analyzed  using  a  Chi-Square  test.  This  is  a  comparison  of  a 
set  of  observed  frequencies  (number  correct  on  first  diagnosis)  with  a 
set  of  expected  frequencies  (expected  number  correct).  The  results  in 
this  section  examine  the  Chi-Square  tests  within  the  3  board  types. 

POWER  SUPPLY  BOARD.  Tables  17  and  18  contain  the  Chi-Square  frequency 
data  for  treatment  conditions  and  student  proficiency  levels.  As  Che 
probabilities  indicate,  there  were  no  significant  frequency  differ¬ 
ences  between  the  independent  variables.  The  analysis  indicates  that 
the  assigned  treatment  condition  and  student  proficiency  level  had 
little  effect  on  troubleshooting  success.  The  success  rate  for  the 
experimental  condition  was  not  different  from  the  success  rate  of  the 
control  conditions. 

FIRST  IF  BOARD.  Table  19  contains  the  Chi-Square  data  by  treatment 
condition  and  indicates  no  significant  differences  in  success  rate  on 
the  first  fault  diagnosis  attempt.  Student  proficiency  level,  howev¬ 
er,  did  have  a  significant  (p<.01)  effect  on  whether  or  not  a  student 
had  a  correct  diagnosis  on  the  fault  diagnosis  (Table  20).  Further 
analysis  indicated  that  the  high  proficiency  students  had  significant¬ 
ly  ( p< .01)  more  correct  first  attempts  than  incorrect,  significantly 
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TABLE  10.  FIRST  IF  BOARD  -  ANOVA  TOTALS  - 
PROBE  TIME  (MINUTES ) 

PROFICIENCY  LEVEL 


TREATMENT  CONDITION 

HIGH 

MEDIUM 

LOW 

Troub les hoot  lag  CAI 

189 

250 

252 

Control  CAI 

111 

26  7 

288 

Control 

106 

193 

185 

TABLE  11.  FIRST  IF  BOARD  -  ANOVA  SUMMARY  -  PROBE  TIME  (MINUTES) 


VARIATION 

SUM  OF 

DEGREES 

VARIANCE 

SOURCE 

SQUARES 

FREEDOM 

ESTIMATE 

F 

Prof ic iency 

3599.96 

2 

1799.98 

2.25 

Treatment 

1418.44 

2 

709.22 

.89 

Interac  t ion 

715.24 

4 

178.81 

.22 

Error 

35920.80 

45 

798.24 

Total 

41654.29 

53 

NOTE:  No  Significant 

i 

Effects 

20 
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TABLE  12.  SECOND  IF  30 A R D  ■  A.NOVA  TOTALS  - 
NUMBER  OF  POINTS  PROBED 

PROFICIENCY  LEVEL 


TREATMENT  CONDITION 

HIGH 

MEDIUM 

LOW 

Troubleshooting  CAI 

186 

213 

180 

Control  CAI 

202 

245 

2  3  y 

Control 

1  1  <3 

L  i  7 

181 

TABLE  13.  SECOND  IF  BOARD  -  ANOVA  SUMMARY  -  NUMBER  OF  POINTS  PROBED 


VAR [AT ION 

SOURCE 

SUM  OF 
SQUARES 

DEGREES 

FREEDOM 

VARIANCE 

ESTIMATE 

r 

Pro  f  ic  mncy 

ro 

U) 

•O 

00 

2 

117.19 

.  26 

T  rea tmen t 

1958.12 

2 

979.06 

2.  3S 

Interact  ion 

CO 

00 

CO 

4 

119.72 

.  2 1 

E  r  r  i  >  r 

18433.63 

45 

411.97 

Tot.il 

21210.07 

5  3 

NOTE :  No  S ign i r icant 

Effects 

P  1 
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TABLE  14.  SECOND  IF  BOARD  -  ANOVA  TOTALS  - 
PROBE  TIME  (MINUTES) 

PROFICIENCY  LEVEL 


TREATMENT  CONDITION 

HIGH 

MEDIUM 

LOW 

Troubleshooting  CAI 

176 

154 

127 

Control  CAI 

198 

344 

204 

Control 

112 

168 

188 

TABLE  15. 

SECOND  IF  BOARD 

-  ANOVA  SUMMARY 

-  PROBE  TIME 

(MINUTES ) 

VARIATION 

SUM  OF 

DEGREES 

VARIANCE 

SOURCE 

SQUARES 

FREEDOM 

ESTIMATE 

F 

Pro  f  ic iency 

1020.30 

2 

510.15 

1.05 

Treatment 

2980,08 

2 

1490.04 

3.05* 

Interac  t  ion 

1972.92 

4 

493.23 

1.01 

Error 

21949.65 

45 

487.77 

local 

27923.05 

53 

NOTE:  *p< . 05 

TABLE  16.  SECOND  IF  BOARD  -  TREATMENT  CONDITION  - 


MEAN  PROBE 

TIME  (MINUTES) 

TREATMENT  CONDITION 

MEAN  TIME 

Troubleshooting  CAI 

25.39 

Control  CAI 

41.44 

Control 

26.00 

22 
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TABLE  17.  POWER  SUPPLY  BOARD  - 
FIRST  FAULT  DIAGNOSIS  ATTEMPT  5Y  TREATMENT  CONDITION 


TREATMEN T  COM D IT  ION 


CORREC  r 
FIRST  ATTEMPT 
YES  NO 


Troubleshooting  CAl 

iO 

8 

Control  CAI 

12 

6 

Control 

11 

7 

“=  .47 
p  =  .79 


TABLE  18.  POWER  SUPPLY  BOARD  - 
FIRST  FAULT  DIAGNOSIS  ATTEMPT  BY  PROFICIENCY  LEVEL 


PROFICIENCY 

LEVEL 


CORRECT 
FIRST  ATTEMPT 
YES  NO 


High 

13 

5 

Med ium 

L0 

8 

Low 

10 

8 

v  '  —  1.40 
p  =  .50 


2  3 
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TABLE  19.  FIRST  IF  BOARD  - 
FIRST  FAULT  DIAGNOSIS  ATTEMPT  BY  TREATMENT  CONDITION 

CORRECT 
FIRST  ATTEMPT 

TREATMENT  CONDITION  YES  NO 


11  7 

9  9 

12  6 

X2=  1.07 
p  =  .59 


Troub leshoot ing  CAI 
Control  CAI 
Control 


TABLE  20.  FIRST  IF  BOARD  - 
FIRST  FAULT  DIAGNOSIS  ATTEMPT  BY  PROFICIENCY  LEVEL 


PROFICIENCY 

LEVEL 


CORRECT 
FIRST  ATTEMPT 
YES  NO 


High 


15  3 


Medium  6  12 

Low  *  117 


X2=  9.36 

p  <.01 
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( p<  .03)  fewer  incorrect  first  attempts  than  the  medium  proficiency 
students,  and  significantly  ( p< .05)  more  correct  first  attempts  than 
medium  proficiency  students. 

SECOND  IF  BOARD.  Tables  21  and  22  indicate  that  the  Chi-Square  anal¬ 
ysis  did  not  reveal  any  significant  ( p<  .05)  frequency  differences  due 
to  treatment  condition  or  proficiency  level.  The  number  of  students 
correct  and  incorrect  on  the  first  diagnosis  attempt  was  not  affected 
by  the  independent  variables. 

DESCRIPTIVE  ANALYSIS  -  TROUBLESHOOTING  LOGIC 

The  objective  of  this  portion  of  the  analysis  was  to  examine  the 
troubleshooting  logic  used  by  the  students,  assess  its  effectiveness, 
and  determine  the  effects  of  the  independent  variables  on  the  Logic 
used.  In  order  to  determine  the"  troubleshooting  strategy  or  logic 
used,  the  sequence  of  probes  taken  to  isolate  the  fault  was  analyzed. 
Each  strategy  used  was  recorded  and  classified  into  one  of  the  follow¬ 
ing  categories: 

a.  HALF-SPLIT  -  The  troubleshooter  successively  tests  the  mid¬ 
point  between  a  known  good  and  bad  signal  or  voltage  until 
the  fauLt  is  located. 

b.  LINEAR  I/O  -  The  troubleshooter  begins  at  the  board  input  and 
tests  the  voltage  or  signal  output  of  each  circuit  sequential¬ 
ly  until  the  faulty  circuit  or  stage  is  found. 

c.  LINEAR  TRACING  -  The  troubleshooter  begins  at  the  board  input 
and  tests  voltage  or  signals  sequentially  until  the  fault  is 
found . 

d.  LINEAR  IN-CIRCUIT  TRACING  -  After  a  particular  circuit  has 
been  isolated  by  any  method,  the  troubleshooter  tests  voltage 
or  signals  sequentially  within  the  specific  circuit  until  the 
fault  is  located. 

e.  RELIABILITY  TESTING  -  The  troubleshooter  successively  tests 
the  least  reliable  untested  component  until  the  fault  is 
found . 

f.  SYMPTOMATIC  -  The  troubleshooter  tests  the  circuit  area  or 
stage  whose  failure  would  cause  the  front  panel  symptoms 
observed . 

g.  RANDOM  -  The  troubleshooter  uses  no  logical  sequence  of 
tes  ts  . 
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TABLE  21.  SECOND  IF  BOARD  - 
FIRST  FAULT  DIAGNOSIS  ATTEMPT  BY  TREATMENT  CONDITION 

CORRECT 
FIRST  ATTEMPT 


TREATMENT  CONDITION 

YES 

NO 

Troubleshooting  CAI 

11 

7 

Control  CAI 

10 

8 

Control 

10 

8 

X2=  .15 
p  =  .93 


TABLE  22.  SECOND  IF 
FIRST  FAULT  DIAGNOSIS  ATTEMPT 

BOARD  - 

BY  PROFICIENCY 

LEVEL 

PROFICIENCY 

LEVEL 

CORRECT 
FIRST  ATTEMPT 
YES  NO 

High  ' 

11 

7 

Medium 

10 

8 

Low  , 

10 

8 

X2=  .15 
p  =  .93 

26 
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h.  LINEAR  COMPONENT  CHECKING  -  The  troubleshooter  sequentially 
Cests  conductivity  of  components  until  a  fault  is  located. 

i.  OTHER  -  The  troubleshooter  appears  co  use  e  logical  tech¬ 
nique,  but  the  technique  cannot  be  identified. 

Many  times  the  sequence  of  probes  indicated  that  the  students  utilized 
more  than  one  technique  to  isolate  a  fault.  The  number  of  strategies 
used  and  the  sequence  of  strategies  were  recorded  until  the  student's 
first  attempt  to  classify  the  fault,  i.e.,  the  first  trip  to  the  LS 
with  a  fault  judgement.  This  section  details  the  analysis  of  trouble¬ 
shooting  strategies  by  board  type. 

POWER  SUPPLY  BOARD.  The  Half-Split  technique  was  utilized  by  most 
students  as  their  first  strategy  (Table  23).  The  number  of  students 
using  Half-Split  first  is  significantly  ( p< .01,  Chi-Square  test) 
greater  than  the  number  using  other  tec'-hniques  first  regardless  of 
treatment  condition.  After  using  the  Half-Split  technique,  students 
tended  to  try  different  techniques,  such  as  Linear  Component  Checking 
and  Random  (Table  24). 

Chi-Square  tests  were  used  to  determine  if  the  number  of  strat¬ 
egies  used  was  affected  by  the  independent  variables  or  if  the  number 
of  strategies  used  affected  troubleshooting  success.  Table  25  con¬ 
tains  the  Chi-Square  frequency  data,  by  treatment  conditions,  for  the 
number  of  students  using  1  to  4  (or  more)  strategies  prior  to  their 
first  trip  to  the  LS  with  a  fault  diagnosis.  Several  students  in  each 
condition  used  more  than  4  strategies,  but  the  majority  of  first  fault 
diagnosis  attempts  occurred  within  4  strategies.  The  analysis  indicat¬ 
ed  that  the  number  of  strategies  used  was  not  affected  by  treatment 
condition.  Table  26  contains  the  total  number  of  strategies  used 
prior  to  the  first  trip  to  the  LS  with  a  fault  judgement,  by  treatment 
condition  based  on  whether  the  first  attempt  resulted  in  a  correct  or 
incorrect  diagnosis.  The  Chi-Square  test  indicated  no  significant 
frequency  differences.  Thus,  the  number  of  strategies  used  was  not 
affected  by  treatment  conditions  and  did  not  affect  success  on  the 
first  attempt  at  a  diagnosis. 

Since  the  Half-Split  technique  was  the  predominant  method  used, 
and  the  method  under  study,  further  analysis  examined  whether  or  not 
students  who  used  it  first  were  more  successful  on  their  first  fault 
diagnosis  than  those  who  did  not  use  Half-Split.  The  data  contained 
in  Table  27  indicate  that  when  compared  to  all  other  strategies  com¬ 
bined,  the  Half-Split  technique  did  not  produce  more  students  correct 
on  the  first  attempt. 

FIRST  IF  BOARD.  The  Half-Split  technique  was  the  predominant  ( p< . 0 i ) 
first  strategy  used  on  the  First  IF  board,  regardless  of  treatment 
condition  (Table  28).  After  Che  Half-Split,  students  tended  to  use 


2/ 


TOTAL  TROUBLESHOOTING  CAI  CONTROL  CAI  CfNTKOL 

STRATEGY  STUDENTS  X  STUDENTS  -  °  NTS  l  STUDENTS 
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TABLE  25.  POWER  SUPPLY  BOARD  -  NUMBER  OF  STUDENTS  USING 
ONE  TO  FOUR  STRATEGIES  PRIOR  TO  FIRST  FAULT  DIAGNOSIS 

STUDENTS  USING 
ONE  TWO  THREE 

TREATMENT  CONDITION  STRATEGY  STRATEGIES  STRATEGIES 


Troubleshooting  CAI  6435 

Control  CAI  6615 

Control  76  i  4 

X2=  2.35 

p  =  .88 

*  Includes  students  using  more  than  four  strategies. 


TABLE  26.  POWER  SUPPLY  BOARD  -  SUCCESS  RATE  BASED  ON  TOTAL 
STRATEGIES  USED  PRIOR  TO  FIRST  FAULT  DIAGNOSIS 

TOTAL  STRATEGIES  USED 
CORRECT  INCORRECT 

TREATMENT  CONDITION  FIRST  ATTEMPT  FIRST  ATTEMPT 


Troubleshooting 

CAI 

32 

16 

Control  CAI 

29 

16 

Control 

19 

19 

X2=  2.81 
p  =  .24 

» 

TABLE 

27.* 

POWER  SUPPLY  BOARD 

-  SUCCESS  RATE 

OF  FIRST  DIAGNOSIS  USING  HALF-SPLIT  FIRST 
VERSUS  ALL  OTHER  STRATEGIES 

CORRECT 
FIRST  ATTEMPT 

FIRST  STRATEGY  USED  YES  NO 


Hal f-Split 

All  Other  Strategies 

X2=  .03 

p  =  .85 


28  19 

5  2 


FOUR 

STRATEGIES'* 


30 


TOTAL  TROUBLESHOOTING  CAT  CONTROL  CAI  CONTROL 

STRATEGY  STUDENTS  Z  STUDENTS  %  STUDENTS  %  STUDENTS 
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Linear  I/O  as  their  .second  strategy,  Linear  Component  Checking  as  the 
third  strategy,  with  no  predominant  fourth  strategy  (Table  29). 

\  Cli  L-Squar.-  test  was  used  to  determine  if  treatment  condition 
at tected  the  number  of  strategies  a  student  used  prior  to  the  first 
crip  to  the  LS  with  a  fault  diagnosis.  Data  in  Table  30  indicate  that 
this  was  not  the  case.  The  total  number  of  strategies  used  by  all 
students  in  each  condition  did  significantly  (p<.05)  affect  success  on 
the  first  trip  to  the  LS  (Table  31).  Further  analysis  found  signif¬ 
icance  between  the  control  CAI  group  and  the  baseline  control  group  on 
incorrect  first  attempts,  i.e.,  the  students  in  the  baseline  control 
group  used  significantly  (p<.Ql)  fewer  strategies  on  incorrect  first 
attempts  than  the  control  CAI  group.  The  Half-Split  technique  did  not 
s igni f icant ly  affect  success  on  the  first  diagnosis  attempt  when 
compared  to  all  other  techniques  (Table  32). 

SECOND  IF  BOARD.  The  first  troubleshooting  strategy  used  on  the 
Second  IF  board  was  predominantly  the  Half-Split  technique  (Table  33). 
The  first  technique  used  was  not  affected  by  treatment  condition.  The 
students  tended  to  use  Linear  Component  Checking  as  the  preferred 
second  strategy,  Half-Split  as  the  third  strategy,  with  no  preferred 
fourth  strategy  (Table  34). 

The  number  of  strategies  used  by  each  student,  prior  to  the  first 
attempted  diagnosis,  was  not  affected  by  treatment  condition  (Table 

35)  and  did  not  affect  the  success  rate  of  that  first  attempt  (Table 

36) .  The  students  using  Half-Split  as  their  first  strategy  were  not 
more  successful  than  students  using  all  the  other  strategies  combined 
(Table  37). 

ET  COURSE  COMPLETION  TIMES 

The  ET  Splice  course  is  self-paced,  and  one  of  the  hypothesized 
effects  of  the  troubleshooting  CAI  was  to  shorten  the  amount  of  time 
required  to  complete  the  subsequent  ET  Splice  curriculum.  Likewise, 
since  the  student  proficiency  level  was  based  on  the  completion  time 
of  the  prerequisite  course,  it  was  predicted  that  the  high  proficiency 
students  would  require  less  time  to  complete  the  course  than  medium 
and  low  proficiency  students  and  that  high  proficiency  students  receiv¬ 
ing  the  troubleshooting  CAI  would  require  less  time  than  all  other 
students . 

Tables  38  and  39  contain  Che  ANOVA  totals  and  summary  data  for  ET 
course  completion  times  (in  total  course  hours).  The  results  indicate 
that  both  proficiency  level  and  treatment  condition  had  a  significant 
(p<.05)  effect  on  ET  Splice  course  completion  time,  but  they  did  not 
have  a  significant  interaction.  The  mean  completion  times  for  stu¬ 
dents  classified  by  each  of  the  independent  variables  are  shown  in 
Tables  40  and  41.  The  LSD  post  hoc  analysis  indicated  that  high 
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TABLE  30.  FIRST  IF  BOARD  -  NUMBER  OF  STUDENTS  US  INC 
ONE  TO  FOUR  STRATEGIES  PRIOR  TO  FIRST  FAULT  DIAGNOSIS 


TREATMENT  CONDITION 

ONE 

STRATEGY 

STUDENTS  USING 

TWO  THREE 

STRATEGIES  STRATEGIES 

FOUR 

STRATEGIES' 

Troubleshooting  CAI 

2 

4  8 

4 

Control  CAI 

1 

7  6 

/, 

Control 

3 

9  6 

0 

X2=  7.30 
p  =  .29 

*  Includes  students 

using  more 

than  four  strategies. 

TABLE  31.  FIRST  IF  BOARD  -  SUCCESS  RATE  BASED  ON  TOTAL 
STRATEGIES  USED  PRIOR  TO  FIRST  FAULT  DIAGNOSIS 


TOTAL  STRATEGIES  USED 
CORRECT  INCORRECT 

TREATMENT  CONDITION  FIRST  ATTEMPT  FIRST  ATTEMPT 


Troubleshooting  CAI 

28 

29 

Control  CAI 

20 

32 

Control 

25 

14 

\2=  3.86 
p  =  .05 


TABLE  32.  FIRST  IF  BOARD  -  SUCCESS  RATE  OF  FIRST 
DIAGNOSIS  USING  HALF-SPLIT  FIRST 
VERSUS  ALL  OTHER  STRATEGIES 

CORRECT 


FIRST 

ATTEMPT 

FIRST  STRATEGY  USED 

YES 

NO 

Half-Split 

30 

21 

All  Other  Strategies 

-3 

1 

1 

X2=  .23 
p  =  .63 
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**  The  total  number  ot  students  is  less  because  not  all  students  utilized  mor 
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TABLE  35.  SECOND  IF  BOARD  -  NUMBER  OF  STUDENTS  USING 
ONE  TO  FOUR  STRATEGIES  PRIOR  TO  FIRST  FAULT  DIAGNOSIS 


STUDENTS  USING 

ONE  TWO  THREE  FOUR 

TREATMENT  CONDITION  STRATEGY  STRATEGIES  STRATEGIES  STRATEGY 


Troubleshooting  CAI 

6  5 

i  6 

Control  CAI 

4  9 

1  4 

Control 

7  8 

i  n 

i.  -A 

y“  =4.00 

p  =  .68 

*  Includes  students 

us  ing 

! 

more  than  four  strategies. 

TABLE  36.  SECOND  IF 

STRATEGIES  USED 

» 

BOARD  -  SUCCESS  RATE 
PRIOR  TO  FIRST  FAULT 

BASED  ON  TOTAL 

DIAGNOSIS 

TREATMENT  CONDITION 

TOTAL  STRATEGIES  USED 

CORRECT  INCORRECT 

FIRST  ATTEMPT  FIRST  ATTEMPT 

Troubleshooting  CAI 

27 

18 

Control  CAI 

26 

20 

Control 

18 

15 

.25 

p  =  .88 


TABLE  37.  SECOND  IF  BOARD  -  SUCCESS  RATE  OF  FIRST 
DIAGNOSIS  USING  HALF-SPLIT  FIRST 
VERSUS  ALL  OTHER  STRATEGIES 


CORRECT 


FIRST  STRATEGY  USED 

FIRST  ATTEMPT 

YES  NO 

t 

Half-Split 

26 

19 

All  Other  Strategies 

5 

i 

3 

.  ■'=  .02 

p  =  .88 
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TABLE  38.  ET  COURSE  COMPLETION  TIME  - 
AN OVA  TOTALS  (HOURS) 

PROFICIENCY  LEVEL 

TREATMENT  CONDITION  HIGH  MEDIUM  LOW 


Troubleshooting  CAI 
Control  CAI 
Cent  r ol 


1693.70 

1676.70 
1582.20 


2255.50 

2381.70 
21 71 .10 


2873.40 
3231  .  10 
2702.10 


TABLE  39.  ET  COURSE  COMPLETION  TIME  -  ANOVA  SUMMARY 


VARIATION 

SOURCE 

SUM  OF 
SQUARES 

DEGREES 

FREEDOM 

VARIANCE 

ESTIMATE 

F 

Prof ic iency 

412774.00 

2 

206387.00 

94. 72** 

T  treatment 

19409.76 

2 

9704.88 

4.45* 

In  te  rac t ion 

9823.52 

» 

4 

2455.88 

1.13 

Error 

98053.20 

45 

2178.96 

Total 

540059.40 

53 

i 


**  p<.01 

'•*  pT  .  0  5 
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TABLE  40.  MEAN  COMPLETION  TIME  BY 
PROt ICiHNCY  level 


PROFIC  IF.NCY 

MEAN 

LEVEL 

TIME 

High 

275.14 

Med ium 

378.24 

Low 

439.26 

TABLE  41.  MEAN  COMPLETION  TIME  BY 
TREATMENT  CONDITION 


TREATMENT 

MEAN 

CONDITION 

TIME 

Troubleshooting  CAI 

379.03 

Concrol  CAI 

404.97 

Control 

353.63 
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prof ic Lency  students  completed  the  course  in  significantly  fewer  hours 
than  the  medium  and  low  proficiency  students  and  that  the  medium 
proficiency  students  required  significantly  fewer  hours  than  the  low 
proficiency  students  (p<.01).  The  LSD  post  hoc  analysis  for  treatment 
condition  indicated  that  the  baseline  control  group  completed  the 
course  in  significantly  less  time  than  the  group  receiving  the  control 
CAI  course  ( p< -01).  There  was  no  significant  difference  in  course 
completion  time  between  the  students  receiving  the  troubleshooting  CAI 
course  and  those  receiving  no  treatment. 

STUDENT  COMMENTS 

The  comments  made  by  students  on  the  troubleshooting  CAI  question¬ 
naire  and  made  to  the  researcher  during  the  performance  tests  are 
summarized  and  outlined  in  the  following  2  sections. 

TROUBLESHOOTING  CAI  COMMENTS 

A  Troubleshooting  Strategy  Questionnaire  administered  to  the  13 
experimental  treatment  subjects,  subsequent  to  their  completion  of  the 
CAI,  indicated  the  following: 

a.  Fifteen  students  felt  the  CAI  unit  on  Isolation  Strategy  was 
the  most  valuable  in  helping  them  learn  effective 
troubleshooting. 

b.  CAI  units  on  System  Flow  Visualization,  System  Visualization 
and  Localization/Fixed  Flow  gave  students  the  least  amount  of 
difficulty. 

c.  All  18  experimental  subjects  felt  the  CAI  unit  on  Feedback 
was  the  most  difficult. 

d.  Of  the  three  media  used  in  presenting  the  experimental  treat¬ 
ment  condition,  all  18  subjects  preferred  computer  interac¬ 
tion;  whereas,  the  least  preferred  mode  of  instruction  was 
split  between  video  instruction  and  the  use  of  a  workbook. 

PERFORMANCE  TEST  COMMENTS.  The  researcher  recorded  pertinent  comments 
made  during  the  troubleshooting  performance  tests  and  the  predominant 
ones  are  as  follows: 


a. 

b. 


Learn  more  in  the  testing  center  than  in  their  assigned 
class. 

Do  not  understand  how  to  read  a  Simpson  Multimeter. 

Prefer  using  a  digital  multimeter. 
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d.  Do  not  understand  how  to  set  up  the  equipment. 

e.  Do  not  remember  to  check  front  panel  symptoms  on  the  trainer. 

f.  Enjoyed  participating  in  the  research  performance  tests,  fee 
a  genuine  concern  regarding  their  performance. 

g.  Feel  confused  while  trying  to  locate  the  fault. 


i 


4  i 
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SECTION  IV 

CONCLUSIONS  AND  RECOMMENDATIONS 

If  the  troubleshooting  CAI  used  in  this  research  had  enhanced  the 
BE&E  School's  training  curriculum,  then  the  students  receiving  that 
CAI  course  shouLd  have  performed  significantly  better  than  the  other 
groups  on  the  performance  tests.  The  research  compared  treatment  and 
control  conditions  in  a  strict  experimental  environment.  Overall,  the 
results  indicate  that  the  troubleshoot ing  CAI  did  not  enhance  perfor¬ 
mance,  and  in  some  cases,  the  baseline  control  group  performed  signif¬ 
icantly  better  than  the  troubleshooting  CAI  group.  The  statistically 
significant  and  non-significant  results  indicate  that  the  control 
group  receiving  no  CAI  course  can  perform  as  well  as  or  better  than 
the  groups  receiving  the  CAI  treatments. 

If  the  troubleshooting  CAI  course  had  given  students  additional 
unique  instructional  material  to  utilize  Half-Split  more  efficiently, 
then  we  would  have  expected  students  receiving  the  troubleshooting  CAI 
to  use  Half-Split  more  than  the  other  groups.  However,  the  analysis 
indicated  that  all  groups  initially  used  the  Half-Split  technique  to 
localize  the  fault  and  that  the  troubleshooting  CAI  did  not  improve 
strategy  usage. 

It  should  be  noted  that  the  CAI  course  resulted  in  significantly 
improved  performance  during  the  pilot  study.  At  the  time  of  the  pilot 
study,  the  Half-Split  troubleshooting  technique  was  not  emphasized  in 
the  School.  In  the  time  period  between  completion  of  the  pilot  study 
and  initiation  of  the  full  study,  the  School  modified  the  curriculum 
toward  a  greater  emphasis  on  the  Half-Split  troubleshooting  technique. 
With  the  increased  emphasis  on  the  Half-Split  technique  in  the  reg¬ 
ular  curriculum,  the  CAI  'course  on  Half-Split  did  not  lead  to  improved 
troubleshooting  performance.  In  fact,  the  significant  ANOVA  results 
indicated  negative  training  effects  on  the  Power  Supply  board  trouble¬ 
shooting  performance.  Of  the  3  boards,  the  Power  Supply  board  was  the 
only  one  with  feedback  loops.  All  students  receiving  the  troubleshoot¬ 
ing  CAI  course  felt  the  unit  on  feedback  was  the  most  difficult.  The 
CAI  instruction  could  have  interfered  with  their  troubleshooting 
efficiency  if  they  did  not  fully  comprehend  its  application. 

The  logic  analysis  and  review  of  the  performance  write-ups  indi¬ 
cate  that  students  use  essentially  the  same  strategies  regardless  of 
additional  training.  However,  review  of  the  logic  analysis  also 
indicated  that  while  students  may  sequentially  probe  points  in  a 
Half-Split  pattern,  they  do  not  always  probe  the  most  logical  points. 
The  Half-Split  technique  requires  the  user  to  probe,  take  a  reading, 
and  then  make  a  judgement  as  to  which  point  will  provide  information 
to  further  localize  the  fault.  Students  using  this  method  did  not 
always  correctly  interpret  the  information  they  were  gathering. 
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Missing  cues,  failure  to  recognize  faults,  and  the  inability  to  distin¬ 
guish  a  good  signal  from  a  bad  one,  were  recurring  problems.  Thus, 
the  Half-Split  technique,  by  itself,  did  not  always  lead  to  the  isola¬ 
tion  of  the  fault.  This  finding  could  indicate  a  rote  probe  sequence 
rather  than  a  logical  usage  of  meter  readings  to  select  the  next 
appropriate  point  to  probe. 

The  treatment  control  condition  received  a  3ASIC  program  which 
should  not  have  affected  performance  on  the  troubleshooting  tasks. 
However,  data  trends  indicate  that  the  group  receiving  the  control  CAI 
performed  with  less  efficiency  than  the  other  groups.  Since  the  BE&E 
School  is  an  intensive  self-paced  program,  students  in  this  group  may 
have  been  affected  by  removal  from  the  electronic  training  environ¬ 
ment.  Removal  from  the  self-paced  program  should  have  affected  both 
CAI  groups,  but  the  control  CAI  may  have  had  additional  interference 
from  2-3  days  concentrated  work  on-  an  unrelated  new  topic. 

The  hypothesized  effect  of  student  *proficiency  level  was  support¬ 
ed  in  the  data  trends  and  significant  results.  As  defined  within  this 
BF.&F  School  for  the  research,  high  proficiency  students  demonstrate 
more  efficient  troubleshooting  performance  than  medium  and  low  profi¬ 
ciency  students.  The  initial  proficiency  levels  set  at  the  beginning 
of  FT  Splice  can  be  used  to  predict  performance  during  the  curriculum 
and  to  predic.  Co  irse  completion  times. 

In  general,  the  research  has  indicated  that: 

a.  The  off-the-shelf  troubleshooting  CAI,  as  used  in  this  re¬ 
search,  does  not  improve  student  troubleshooting  performance. 
The  CAI  and  School  curricula  should  be  reviewed  to  determine 
if  utilization  at  another  training  stage  will  enhance 
performance . 

b.  Review  of  the  curriculum  and  CAI  should  be  made  to  determine 
if  a  strategic  troubleshooting  course  developed  specifically 
for  BE&E  School  would  enhance  performance. 

c.  Review  of  curriculum  and  student  performance  should  be  made 
to  determine  if  students  understand  the  Half-Split  technique 
or  are  probing  points  by  rote  memory. 

d.  Student  proficiency  level  (based  on  BE&E  completion  times) 
can  be  used  to  predict  performance  in  ET  Splice  School.  Low 
proficiency  students  should  be  given  tutorial  assistance  to 
improve  their  troubleshooting  performance. 
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GLOSSARY 


AF.T 

Actual  Equipment  Trainer 

A  NOVA 

Analysis  of  Variance 

3E&£ 

Basic  Electricity  S  Electronics  School 

CAT 

Computer  Assisted  Instruction 

CM  I 

Computer  Managed  Instruction 

ET 

Electronic  Technician 

First  IP' 

! 

First  Intermediate  Frequency  3oard  (Medium 
Complexity) 

L.S 

Learning  SuDervisor 
'  *  » 

LSD 

Least  Significant  Difference 

Power  SuppLy 

Power  SuppLy  Board  (High  Complexity) 

Second  IF 

Second  Intermediate  Frequency  Board  (Low 
Comp  lex ity ) 
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